Generalized Time Warping for Alignment of Human Behavior

Temporal alignment of human motion performing similar activities has been a topic of recent interest due to its many applications in animation, tele-rehabilitation or activity recognition. This paper presents generalized time warping (GTW), an extension of dynamic time warping (DTW) for temporally aligning multi-modal sequences from multiple subjects performing similar activities. GTW solves three major drawbacks of existing approaches based on DTW: (1) GTW provides a feature weighting layer to adapt different modalities (e.g., video and motion capture data), (2) GTW extends DTW by allowing a more flexible time warping as combination of monotonic functions, (3) unlike DTW that typically has a quadratic cost, GTW has linear complexity in terms of the length of the sequence. Experimental results demonstrate that GTW can efficiently solve the multi-modal temporal alignment problem, and outperforms state-of-the-art methods for temporal alignment of signals with the same modality.